Skip to content

fix(operator): report missing gateway CRDs instead of claiming Ready - #173

Merged
dcmcand merged 2 commits into
mainfrom
fix/llmmodel-surface-missing-crd
Nov 20, 2026
Merged

dcmcand merged 2 commits into
mainfrom
fix/llmmodel-surface-missing-crd

Conversation

@dcmcand

@dcmcand dcmcand commented Aug 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #109. Rewritten on top of main: #190 landed after this was first opened and already gave SecurityPolicies their own SecurityPoliciesReady condition and early apply, so this version builds on that instead of folding policies into the endpoint conditions.

The bug

An LLMModel whose InferencePool or AIGatewayRoute couldn't be applied reported phase Ready. The reconciler logged the error and moved on, so the only trace was the operator log, and the visible symptom was a 404 or 500 at request time against a healthy-looking model.

What changed

  • InferencePoolReady, ExternalEndpointReady and InternalEndpointReady conditions on LLMModel, with the reasons PassthroughModel already uses: GatewayCRDUnavailable for a missing CRD, ApplyFailed otherwise, EndpointDisabled for an endpoint switched off in the spec.
  • On an LLMModel the endpoint conditions cover the route only. The policies stay in SecurityPoliciesReady, applied before the model is serving so key revocation isn't delayed.
  • A failed apply turns Ready into Degraded (pods serve, nothing routes) and requeues: 15 seconds for a missing CRD, one minute otherwise. Pending, Downloading, Starting and Error are left alone.
  • updateStatus sets conditions one at a time so SecurityPoliciesReady survives the final status write.
  • Condition constants and conditionFor/disabledCondition move from the PassthroughModel controller to conditions.go, with new gatewayRequeue, hasGatewayFailure and phaseWithGatewayFailure helpers. PassthroughModel behavior is unchanged.
  • architecture.mdx and configuration.mdx describe the new conditions and drop the Converge LLMModel reconciler onto the PassthroughModel gateway-apply failure handling #109 caveat.

⚠️ Behavior change

A model with broken routing that used to report Ready now reports Degraded. Dashboards and alerts keyed on phase will see it.

How to test

  • cd operator && make test (with AI_GATEWAY_CRD_DIR as in CI): all packages pass.
  • conditions_test.go: table tests for conditionFor, gatewayRequeue/hasGatewayFailure (a disabled endpoint isn't a failure; a missing CRD sets the interval) and phaseWithGatewayFailure.
  • gateway_conditions_test.go: a fake client rejects AIGatewayRoute applies with Forbidden while policies succeed. The model goes Degraded with ApplyFailed, requeues after a minute, keeps SecurityPoliciesReady=True, and returns to Ready with all conditions Applied once the failure clears.
  • envtest spec "Missing gateway CRDs": envtest has no AI Gateway or Inference Extension CRDs, so a serving model reports Degraded with all three gateway conditions GatewayCRDUnavailable, and SecurityPoliciesReady is still present.
  • Mutation-checked: removing the phase downgrade, replacing the condition list in updateStatus, dropping the requeue, or swallowing route errors each fails a test.
  • golangci-lint v2.15.0: no new findings (51 before and after, all in untouched lines). CI runs the pinned version.
  • cd docs && npm test: 31/31.

…ming Ready

LLMModel logged and skipped a failed InferencePool or AIGatewayRoute
apply, then reported Ready. A missing AI Gateway or Inference Extension
CRD left a model that looked healthy and returned 404 or 500 at request
time, with nothing in its status to explain it.

Report each apply as a condition, InferencePoolReady,
ExternalEndpointReady and InternalEndpointReady, using the reasons the
PassthroughModel reconciler already uses: GatewayCRDUnavailable for a
missing CRD, ApplyFailed otherwise, EndpointDisabled for an endpoint the
spec turned off. A failure turns Ready into Degraded and requeues the
reconcile, after 15 seconds for a missing CRD or a minute otherwise. On
an LLMModel the endpoint conditions cover the route only; the policies
keep their own SecurityPoliciesReady condition, applied earlier in the
reconcile so key revocation isn't held up while a model starts.

updateStatus now sets conditions individually instead of replacing the
list, so SecurityPoliciesReady survives the final status write.

The shared condition constants and helpers move from the PassthroughModel
controller to conditions.go.

Closes #109.
@dcmcand
dcmcand force-pushed the fix/llmmodel-surface-missing-crd branch from b903c01 to 2630753 Compare November 19, 2026 14:15
@github-actions

Copy link
Copy Markdown

📄 Docs preview for fix/llmmodel-surface-missing-crd:
https://fix-llmmodel-surface-missing.llm-serving-pack.pages.dev

@jbouder jbouder left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change: a missing SecurityPolicy CRD still leaves the model Ready.

SecurityPoliciesReady is patched earlier in the reconcile and is not in
the conditions phaseWithGatewayFailure checks, so a model whose routes
applied but whose SecurityPolicies could not (Envoy Gateway's CRD not
installed) still reported Ready. Downgrade Ready to Degraded when the
policies are pending; the existing 15s requeue retries them.

The missing-CRD test now checks the phase is Degraded while the CRD is
missing and Ready again once it is installed.
@dcmcand
dcmcand requested a review from jbouder November 20, 2026 13:57

@jbouder jbouder left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. LGTM.

@dcmcand
dcmcand merged commit 7dda98b into main Nov 20, 2026
21 checks passed
@dcmcand
dcmcand deleted the fix/llmmodel-surface-missing-crd branch November 20, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converge LLMModel reconciler onto the PassthroughModel gateway-apply failure handling

2 participants